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Abstract 

The Kolmogorov-Sinai entropy in the sense of Tsalhs under Bernoulh shifts was 
obtained by Meson and Vericat [J. Math. Phys. 37, 4480(1996)]. In this paper, we 
propose a revised generahzed Kolmogorov-Sinai-q entropy under Markov shifts. The 
form of this generahzed entropy with factor q is nonextensive. The new generahzed 
entropy contains the classical Kolmogorov-Sinai entropy and Renyi entropy as weh 
as Bernoulh shifts as special cases. Applying the generalized entropy we discuss its 
approximate behavior qualitatively, the entropy rate and the sensitive value q* of 
the nonextensive parameter g, which may exit in the interval (-2,2) nearby where 
the generalized entropy return to the classical K-S entropy. 
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1 Introduction 



Both entropy theories of Boltzmann-Gibbs (BG) and Shannon are two of the 
most important foundations of statistical physics and modern information 
theory respectively, the Kolmogorov-Sinai (KS) entropy [TP] in the dynamic 
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systems make a great progress, which estabhshes a milestone to modern dy- 
namic systems and affects the development of mathematical physics since 
then. Many concepts characterized complex multifractal systems such as di- 
mension, dimension spectrum[3j, entropy, generalized entropy [4j5 j (such as cor- 
relation entropy, particularly, Renyi entropy[6]), Lyapunov exponent, topolog- 
ical pressure[7], etc., have gained very important applications in physics. More 
recently, a generalized entropy to Boltzmann entropy in physical systems is 
proposed by Tsallis[8j, which has the nonextensive property and hence may 
be a good description for some complex systems with multifractal properties. 
The Tsallis entropy has been applied to various physical phenomena which 
may deviate from equilibrium state for instance Pll0fllfl2lll3fl4] to partic- 
ularly edge phenomena of chaotic threshold in one- dimensional dissipative 
systems [TSlllGf 17] . It is worthy to concern the progress of works in two branches 
to physics at present. On the one hand, the nonextensive forms of the Tsal- 
lis entropy in some important branches of mathematics have been developed. 
A generalized entropy in the Tsallis sense for the dynamic systems has been 
presented by Meson and Vericat first time[T8]. The generalization does what 
Kolmogorov did for Shannon entropy under the Bernoulli scheme, and obtains 
a mathematical parallel which is proved as an isomorphism invariant (THUS] . 
On the another hand, the vague relation between the EG entropy of physics 
and the KS entropy of dynamics has been clarified gradually by works of many 
authors [20] as research of Tsallis entropy. The KS entropy can be interpreted 
with a rate of physical entropy production per unit time. Thus the position of 
KS entropy of dynamics in the application of statistical physics becomes more 
important. 

The Meson- Vericat 's definition of KSq entropy is nice to consider the property 
of power distribution which is a common character of objects described with 
Tsallis entropy. Meanwhile that Tsallis entropy can smoothly recover to BG 
entropy is a well-known advantage. However the Meson-Vericat's definition of 
KSq entropy seems to be lack of the advantage, i.e. it can not recover KS 
entropy when g = 1. To this aim we would revise Meson-Vericat's definition 
and make it more complete. Secondly, the Bernoulli shift is a basic scheme 
of great importance in measure-preserving dynamic systems. When systems 
to be studied are extended and enlarged, it is necessary to exploit rather 
complex shifts beyond Bernoulli ones. So the another aim of this paper attempt 
to extend the work of Meson and Vericat from the Bernoulli scheme to the 
Markov scheme, and present a generalized mathematical form of KS entropy 
in the sense of Tsallis under Markov shifts. The generalized entropy should 
be consistent in mathematics with both cases of the classical KS entropy and 
the Kolmogorov-Sinai-like (KSq) entropy under Bernoulli shifts and our work 
may be a approach to reach this goal. Finally, interrelation of both Renyi and 
Tsallis entropy is interested and focused in statistical physics community now. 
It is one important fact in statistical physics that obtaining same distribution 
under maximal entropy principle are equivalent because they are monotonic 
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each other. We will see also that there is similar situation in the level of 
abstract dynamics. 



2 The Revised K-S-q entropy 

Now we consider the generalized Markov shift in the measure-preserving dy- 
namic system. Let > 2 be a fixed integer which is cardinal of the symbolic set 
Y = {0,1, Let a direct product space (X, i3) = n^^{0, 1, . . . , 

be a space of doubly infinite, where B is the subset family equipped the 
product cr-algebra on the space X. The shift transformation T : X ^ X 
is T{xn} = {xn+i} For each natural number n the cylinder set 

{i^^rJx,=^,, for IjI <n} (z, G F, ) (1) 

forms the basic set family with the product a-algebra. We can introduce a 
unique probability measure m on (X, B) as 

m{{xi e X\xi = io, xi+i = ii,..., xi+n = in-i}) = Pn{io, ii,---, in-i), (2) 

where probability pn{io,H, ■ ■ ■ jin) is a positive real number. According to 
Kolmogorov consistency theorem [2 Ij the probability measure m in ([2]) can 
be extended to the entire space (X, B) . The Markov shift is a quadruple 
{X, B, m, T) and one of the important cases of measure-preserving dynamic 
systems. If we may endow some explanations to it, X is regarded as a sam- 
ple space made up of the whole results of individual random experiments. 
Then the product cr-algebra B on X and the probability measure m associ- 
ated with the cylinder set are the standard frame for the random experiments. 
For one individual experiment the sample leads to the probability of the cylin- 
der pn{iQ,ii, . . . ,in). Different from the Bernoulli scheme, if we suppose that 
the outcomes of every experiment are not independent, the transition prob- 
ability Pij > between both states depends only on the states 
Then we have an irreducible non-negative k x k stochastic matrix P = {Pij} 
(Pij > 0, i,j e Y) with its row normalizing condition S^^gPij = 1 for each i,j 
and there is a unique positive left vector p = (po, • • • ,Pk-i) which satisfy the 
invariance probability 

p, = TltZ^p^P,^. (3) 

We will assign the transition with Markov property to the probability of the 
cylinder as 

with the initial probability vector i.e. Po(^o) = (PojPi? • • • iPk-i) "we obtain a 
two-sided (p, P) Markov shift. The measure preserving is guaranteed by two 
normalizing conditions: Tl^zlPij = 1 and Siggypol^o) = 1- If the transition 
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probability is independent of the initial state i, i.e., taking Pjj- = pj, then the 
probability of cylinder (jlj) becomes 

Pn{io,il, ■ ■ ■,in-l) = PioPiiPi2 ■ ■ -Pin-i- (5) 

The p-Bernoulli shift is a special case of the (p, P) Markov shift. 

According to Shannon's information theory [22], if we already know a point 
X E X belongs to some fixed set of partition A = {Aq,Ai, . . . ,Am-i}, the 
quantity of information is 

■m— 1 

^i(^) = -E/^(^OiogM^O, (6) 

i=0 

where partition A should satisfies the normalizing condition /^(^j) = 1- 

This entropy possess additivity. Tsallis presented a generalization entropy|8j 
of the physics entropy as 

H,{A) = i<i-^r\^-T.HA,)]A, (7) 

where q is an arbitrary real number, and the entropy has pseudo-additivity 

E,{A + B) = H,{A) + H,{B) + (1 - q)H,{A)H^{B) (8) 

when two independent subsystems A and B form one system. The Tsallis 
entropy recovers the Shannon entropy when g = 1. We now define a gen- 
eralized entropy parallel to Ref. [TS] for the finite partition partition A = 
{Aq, Ai, . . . , Am-i} of the sample space. Based on Meson & Vericat's definition [18], 
a new revised definition of generalized entropy( KSq entropy) of the measure- 
preserving transformation T is 

hKSg{T) = {q- - exp[(l - q)hKs,{T)]], (9) 

hKSq(T) = sup hKSq{-A,T), (10) 
^CB,^ finite 

-1 n— 1 

hKsM, T) = \hu -{(1 - g)-i log[l + (1 - q)H,i V T-^A)]}. (11) 

where the supremum in ffTOl) is taken over the set B formed with all finite 
sub-cr-algebras A. It is enough to physics, if not, just taking over all countable 
partitions like current monographs [23j. After getting rid of an infinite point 
partition entropy we can still work on the finite entropy, because the finite 
entropy rate is a necessary condition to statistical mechanics. Note that the 
partition entropy Hq in (ITT!) takes the form of Tsallis entropy. The revised def- 
inition improves the nonextensive property of the parameter q. Since q is any 
real number, the case q = I will be imposed continuity. By using L'Hospital 
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rule the definitions of nonextensive entropy (pi) (fTOj) (fTT]) recovers smootlily to 
tliat of classical K-S extensive entropy 

hKsiT) = hKSliT) = hKsiiT), (12) 

hKsi{T)= sup hKsiiAT), (13) 

1 " 

hKsM.T) = lim -Hr{ \J TA). (14) 

n-^oo n 

i=—n 

Thus the classical definition of KS entropy is contained within the the nonex- 
tensive parameter g, so the revision enhance a little completeness to the defi- 
nition of Meson and Vericat^^. We will see that some conclusions of classical 
KS entropy should also be contained in the generalized KSq entropy below. 



3 the K-S-q entropy of Markov Shifts 

We now come to seek the form of a generalized Markov shift. There exist 
many generalized Markov shifts on the basis of the classical Markov shift, we 
want to seek such a generalized Markov shift which can generalize KS entropy 
according to the form of Tsallis entropy. We assume that there is a family of 
pseudo-stochastic matrices with the real parameter q: {Pj^}j j^y, P^- > and 
still satisfies the condition S^ZoPjj = 1. One recognizes in the analysis of mul- 
tifractal that the Tsallis entropy is analogy to g-dimension in its mathematical 
form [2^1125] . It is necessary to introduce a new L^-probability Tj-. 

= (Zp^P!,Y'\ (15) 

which will be used as the partition probability of the generalized Markov shift. 
Of course, the generalized probability can play the role of pj in the classical 
Markov shift, when q approaches to 1, it has pj and recovers to the 

classical Markov shift. Particularly, if Pij = pj, then r| is just the partition 
probability of the generalized Bernoulli shift defined by Meson and Vericat [T8] . 
So the partition probability contains the generalized Bernoulli shift as a special 
case. It will lead to the KSq entropy of the Markov shift we wanted. 

We now calculate the KSq entropy of generalized Markov shift. With the defi- 
nition of product cr-algebra there exits a canonical generator Q = T'^A. 
For the generalized entropy the calculation is analogy to Bernoulli scheme [18j . 
Considering a typical element of the refinement (^(V"=o^ 7"^'^) 

= {{Xn} : Xq = io,Xi = ii, . . . , Xn-l = in-l}, < ij < k - 1 
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and using the generalized Markov shift and the Lg-probabihty rj in the pre- 
vious section, we will assign the new probability measure to the partition A, 
i.e. = Cj, we can take the Lg-probability partition in fill I) 



= r{xo = jo)r{xi = ji) ■ ■ ■ r{xn-i = jn-i) 



(0 < jo, ji, • • • , jn-1 < A; - 1). 



So the generalized refinement entropy of probability space can be derived by 
the probability Tj 



n-l 



1 



fc-1 k~l 
j=0 i=0 



(16) 



Put (fT6l) into ([9]), (ITO!) and (fTTj) . then we have the generalized KSq entropy of 
transformation T under Markov shifts: 



fc-i fc-i 

1 - E E P.P^j 

hKs,{T) = (17) 

q- 1 

= -Y.11p^p>^p^3- (18) 

where 

ln,x = ^^^ (g^l). (19) 
1 - q 

We see that fllSp returns to the original form of KS entropy when q ^ 1 

k—\ k—1 

hKsi{T) = - J2J2p^P^j log^^„ (20) 

i=0 j=0 

and since the Bernoulli shift is a special case of the Markov shift, when Pij = 
Pj, (ITS]) becomes 

k-l 

I - vt 

K{T) = (21) 

q-1 

This expression recovers the generalized entropy of the Bernoulli shift obtained 
by Ref . [TH] . Thus f[T5]) contains classical KS entropy and generalized Bernoulli 
KSq entropy. Furthermore, it is noted that for the generalized Markov scheme 
both the entropy of T and entropy of T with respect to A do not equal. 

hKSgiT)^hKsM^T). (22) 
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Using the simple calculation with definition (ITT]) for (q ^ 1) we have 

k—l k—1 

This is just the well-known Renyi entropy[6] in the Markov scheme. The en- 
tropy of transformation T with respect to partition A is one part of the def- 
inition of KSq entropy, i.e. The KSq entropy contains Renyi entropy as a 
component in context of dynamic systems. When hKSqi-^, ^) is maximized by 
supremum through (ITOl) and ([9]), it leads to KSq entropy. This implies an uni- 
fication of both Tsallis and Renyi entropy in KSq form, naturally, it includes 
a mergence of two parameters q and f3 which should not be confused in the 
viewpoint of current issue, because there is a great difference in the additivity 
of both. Particularly, we have 

hni hKSq(T) = lira hxsqiA, T), (23) 

5^1 q^l 

it shows that the both are generalizationes of the KS entropy along the line of 
abstract dynamics although Renyi and Tsallis entropy are different in the sense 
of Kolmogorov-Nagumo average [26]. For the Bernoulli scheme, as it is a spe- 
cial case of the Markov scheme, hxsqiT) does also not necessarily agree with 
the entropy hKSq{-A,T) by ( !22|) . even hKSq(T) = i?q(p)[18] for the Bernoulli 
scheme of arbitrary q. According to the revised definition (ITTi) we still have 
hKSq{-A.,T) = (1 — q)~^ Pj\ to be Renyi entropy in Bernoulli scheme, 

so the recover fl23|l must also be available. 

Isomorphism equivalent class of the KSq entropy in Ornstein's sense is a com- 
plex problem for Markov shifts. We will not discuss it since it is not a complete 
invariant for Kolmogorov automorphism. However, isomorphism of theKSq 
entropy for Bernoulli shifts will be able to discuss as same as Meso n and 
VericatyiSj. 



4 Application 

The asymptotic behavior of KSq entropy is interesting, for it ties closely with 
the physical meaning of the non-extensive parameter q. We discuss it with 
Beck's frame p7] of the statistics of dynamic symbol sequences. For a given 
map T the Shannon entropy H 

k~l 

H{fi,A,n) = - /i(zo,n, • • • ,«n-i) ln/i(io,»ii, • • • 

io,h,...,i„~i=0 

where ii{io, ii, ■ ■ ■ , in-i) = fJ'iAo n T~'^Ai^ fl ■ ■ ■ n T""+Mj^_ J. The study of 
the asymptotic behavior of the dynamic iterate sequences involves the entropy 



7 



rate per unit time 

h(a,A) = lim (H/n). 

n— >co 

The quantity h{fi,A) depends still on the partition A. Maximizing it over all 
possible partition set {A} in the phase space 



h{fi) = sup h{fx, {A}). (24) 
{A} 

The K-S entropy of the physical version is obtained, which is a global quan- 
tity to describe chaos solely, when we focus on the numerical calculation, 
the supremum can be replaced by the maximization if the generating parti- 
tion exists[28]. In order to clarity the vague connection between the physical 
entropy and the KS entropy. Latora and Baranger[20] calculate the entropy 
rate [15] 

Slit) 

Ki = lim lim lim , (25) 

t^OO W^QO N^QO t 

where the iterate number t = n,W is the cell number of partition of the phase 
space worked. Initially putting N{N ^ W) points into one cell randomly, 
then the occupancies {iVj(t)} of all cells can calculate probabilities piit) = 
Ni{t) / J2T=Q ^ii't) numerically. It is found by the numerical experiment that if 
the S(t) is given by 

w 

i=0 

then there exists a linear value of the entropy rate which is just the KS 
entropy [20 j. To describe the edge-of-chaos, the Boltzmann-Gibbs-Shannon en- 
tropy above is not be appropriate. The non-extensive entropy must be applied [29]. 
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lim lim lim (26) 

t^oo W^oo N^oo t 

For the Markov scheme here we have the non-extensive entropy to be 

1 - E r-Ut) 

S,{t) = ^ . (27) 

The asymptotic behavior of the nonextensive entropy ( |2711 is monotonic de- 
crease, as the parameter q approaches to infinity i.e. liniq^-ochKSqiT) oo 
and Imiq^oohKSqiT) — >• (even limg^oolngX oo). It is allowed that there 
may exist a special sensitive value q* ^ 1 between both entropic values (cxd, 0). 
The time evolution of the non-extensive entropy will sharply turn when q ap- 
proaches g*, which is responsible to the power-law with complicated case in 
nonextensive statistics such as edge-of-chaos and the chaos nearly by eventual 
periodic orbits. The latter is another power-law example to be discussed. It 
is noted that the continuous at g — 1 guarantees the nonextensive entropy 
is smooth and applicable. Since it has recognized that the KS entropy is a 
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production rate of Boltzmann entropy pre unite time in physical systems, it 
leads to that this production rate will decay as q increases. So the case g < 1 
will have more high production rate, it enhances the randomness of system, 
and the case g > 1 decreases the randomness in the Markov shift. Thus the 
parameter q may be a factor which can control and influence random, irregu- 
lar chaotic behavior. For the Bernoulli case the conclusion is the same as the 
Markov one in qualitative. Finally, in order to estimate q* we discuss another 
special value g = 0. For the the entropy of T with respect to {^}, the Renyi 
entropy is just the topological entropy hKso{.-^^T) ^ log A;, which is smaller 
than the K-S-q entropy hKsoiT) ^ {k — 1), i.e., h^sqi-^yT) < h^sqiT) will 
always hold. We may reasonably estimate that the sensitive value q* should 
locate in a symmetric interval with the center at g = and the two symmetric 
terminal points are (—2,2), i.e., —2 < g* < 2. The prediction is right for the 
edge-of-chaos[29j. To confirm the property of the generalized KSq entropy, 
we have applied it to eventual periodic orbits and their edge-of-chaos with 
symbolic dynamics of the unimodal transformation [30] and numerical results 
show us that the KSq entropy indeed has good monotonic property with g 
and sharply turn at the interval nearby where it recovers to KS entropy at the 
limit g ^ 1. 



Markov shifts are available for use of physics. Applications of the theory of 
Markov shifts widely spread in various practical fields, for instance, continuous 
Markov processes by the time shift operator and discrete Markov chains. Many 
practical applications are discussed in details by many books, for instance in 
the book of Bharucha-Reid[31|, such as the growth of populations, mutation 
progress, spread of epidemics, theory of gene frequencies in biology; theory of 
cascading progress in physics; theory of cosmic rays in astronomy; chemical 
reaction kinetics in chemistry; theory of queues, and so on. Particularly, it is 
interesting in the symbolic dynamics associated with edge of chaotic phenom- 
ena in one-dimensional dissipative systems where the topological Markov shift 
on discrete chain plays an very important role. 
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